Multi-mode optical navigation

ABSTRACT

A system and method is provided for selecting a light source in a pointing device such as a mouse. The selection of the light source may be based on attributes of a received image, which are in turn based on reflected light received at the pointing device from the tracking surface. Because the attributes of the receive image are related to characteristics of the tracking surface over which the pointing device is moved, an illumination source appropriate for a particular surface type can be chosen.

BACKGROUND

Measuring motion in two or more dimensions is extremely useful innumerous applications. Computer input devices such as mice are but oneexample. In particular, a computer mouse typically provides input to acomputer based on the amount and direction of mouse motion over a worksurface (e.g., a desk top). Many existing mice employ an imaging arrayfor determining movement. As the mouse moves across the work surface,small overlapping work surface areas are imaged. Processing algorithmswithin the mouse firmware then compare these images (or frames). Therelative motion of the work surface can then be calculated by comparingsurface features common to overlapping portions of adjacent frames.

Imaging can be performed in various ways. One imaging method involvesgrazing illumination in which light from an LED strikes a surface at arelatively shallow angle. The LED light beam striking the surface at therelatively shallow angle is reflected back to a light sensor. Featureson the surface generate shadows, and an image frame composed of suchshadows can be compared with other image frames to calculate directionand amount of motion.

Another imaging method utilizes specular illumination. In this method, ahighly focused LED or a coherent light source (e.g., a laser) strikes asurface at a less shallow or “deep” angle with respect to the surfacebeing imaged.

Each of the above described techniques works better for certain types ofsurfaces. For example, a specular light source providing an incidentlight beam at a deep angle with respect to a surface usually providesbetter tracking images if the surface is glossy or highly reflective.However, specular-image tracking may not work as well if the surface isnon-glossy or not highly reflective. Conversely, tracking based onimages from a grazing illumination light source is generally moreeffective if the surface is not highly reflective and/or relativelyrough.

SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

In at least some embodiments, an input device can select an illuminationsource and corresponding tracking algorithm for generating an image andproviding improved tracking for a particular surface. In one example, aninput device is provided for tracking a tracking surface and includes acontroller for selecting a light source based on an attribute of areceived image from a tracking surface. For example, the input devicemay include at least two light sources for illuminating a surfaceexternal to the input device and track motion of the input device overthe surface. Also, the input device may select a light source forproviding illumination based on a detected attribute of the surface. Inone example, the attribute includes one of image contrast andillumination.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings and in whichlike reference numerals refer to similar elements and in which:

FIG. 1 shows a computer mouse according to at least one exemplaryembodiment.

FIG. 2 is a diagram illustrating the tracking surface shown in FIG. 1.

FIG. 3 is a partially schematic block diagram of an integrated circuitof the mouse in FIG. 1.

FIG. 4 is a partially schematic diagram of an array in the mouse of FIG.1.

FIG. 5 is a flowchart that illustrates an example of determining anillumination method and tracking algorithm according to at least oneembodiment.

FIG. 6 is a flowchart illustrating another example of determining anillumination method according to another embodiment.

FIG. 7 is a block diagram illustrating an example of an apparatus forselecting an illumination method according to one embodiment.

FIG. 8 is a block diagram illustrating another example of an apparatusfor selecting an illumination method in an input device according to oneembodiment.

FIG. 9 is a block diagram of a mouse containing at two separate lightsources and two separate receiving arrays.

DETAILED DESCRIPTION

Various exemplary embodiments will be described in the context of atracking system used to measure movement of a computer mouse relative toa desk top or other work surface. However, the invention is not limitedto implementation in connection with a computer mouse. Indeed, theinvention is not limited to implementation in connection with a computerinput device.

FIG. 1 shows a computer mouse 200 according to at least one exemplaryembodiment. Computer mouse 200 includes a housing 212 having an opening214 formed in a bottom face 216. Bottom face 216 is movable across awork or tracking surface 218. For simplicity, a small space is shownbetween bottom face 216 and work surface 218 in FIG. 1. In practice,however, bottom face 216 may rest flat on surface 218. Located withinmouse 200 is a printed circuit board (PCB) 220. Positioned on anunderside of PCB 220 is a first light source 223. In the embodiment ofFIG. 1, first light source 223 is an LED. The first light source 223directs a beam 225 at a portion of surface 218 visible through opening214. Beam 225, which may include light of a visible wavelength and/orlight of a non-visible wavelength, strikes tracking surface 218 and isreflected into an array 226 of a motion sensing integrated circuit (IC)228.

Beam 225 from the first light source 223 provides grazing illuminationsuch that the beam 225 strikes the tracking surface 218 at a relativelyshallow angle (i.e., angle A as illustrated in FIG. 1). Angle A is arelatively small angle such that the beam 225 “grazes” the trackingsurface 218. For example, angle A may be approximately 20°. In anotherexample, angle A may have other values less than 30° (e.g., 28°, 26°,25°, 23°, 20°, 19°, 18°, 17°, 16°, 15°, 14°, 13°, 12°, 11°, 10°, 9°, 8°,7°, 6°, 5°, etc.). Angle A could also be greater than 30° (e.g., 31°,32°, 33°, 34°, 35°, 36°, 37°, etc.). The grazing illumination from thefirst light source 223 creates an image pattern that is reflected intoand detected by the array 226. For example, if the tracking surface 218is a non-glossy surface, the first light source 223 may generate shadowsamong surface features of an illuminated portion of tracking surface218.

In the embodiment of FIG. 1, grazing light source 223 is an LED. Inother embodiments, a laser can be used as a grazing light source. Adiffuser, spreader or other optical elements can be interposed between agrazing light source (whether LED or laser) and the tracking surface.

The mouse 200 further includes a second light source 222 positionedunderneath the PCB 220. In the embodiment of FIG. 1, the second lightsource 222 is a vertical cavity surface emitting laser (VCSEL). Whenactivated, the second light source 222 directs a beam 224 at a portionof surface 218 visible through opening 214. In the embodiment of FIG. 1,the second light source 222 is positioned to emit light beam 225 tostrike a point on the tracking surface 218 that is coincident with beam225 from the first light source 223. However, this need not be the case.Similar to beam 225 from the first light source 223, light beam 224 fromthe second light source 222 may include light of a visible wavelengthand/or light of a non-visible wavelength. Light from beam 224 strikestracking surface 218 and is reflected into array 226 of motion sensingintegrated circuit (IC) 228.

Light beam 224 from the second light source 222 strikes the trackingsurface 218 at an angle that is different from the angle formed by thelight beam 225 from the first light source 223. As set forth above, thelight beam 225 from the first light source 223 is a grazing light beamand forms an angle A that is relatively shallow relative to the trackingsurface 218. The light beam 224 from the second light source 222 strikesthe tracking surface 218 at an angle B that is larger than angle A. Inother words, light beam 224 from the second light source 222 is morenormal (i.e., closer to being perpendicular) to the tracking surface218. However, light beam 224 from the second light source 222 need notactually be perpendicular to the tracking surface 218. As illustrated inFIG. 1, light beam 224 may in some embodiments be within approximately5-30 degrees of normal (perpendicular) to the tracking surface. In oneembodiment, the light beam 224 from the second light source strikes thetracking surface such that the light beam 224 forms an angle with thenormal axis (i.e., perpendicular orientation with respect to thetracking surface 218) that is approximately 20°. In other embodiments,the angle relative to the normal axis has other values less than 30°(e.g., 28°, 26°, 25°, 23°, 21°, 19°, 18°, 17°, 16°, 15°, 14°, 13°, 12°,11°, 10°, etc.). In yet other embodiments, the angle with the normalaxis is greater than 30° (e.g., 31°, 32°, 33°, 34°, 35°, 36°, 37°, 38°,39°, etc.).

The second light source 222 provides specular illumination in which thelight reaching array 226 has, when the portion of tracking surface 218illuminated by light source 222 is sufficiently glossy or otherwise hasthe proper characteristics, a high frequency pattern of bright and darkregions. That high frequency pattern is the result of the specularillumination and reflection from the surface variations of the trackingsurface, and generally causes the intensity of light falling ondifferent parts of array 226 to vary. These patterns allow trackingmotion of surface 218 relative to mouse 200.

In the embodiment of FIG. 1, specular light source 222 is a VCSEL. Othertypes of laser sources could be used for the specular light source.Indeed, specular light source 222 is not a laser in some embodiments.For example, some embodiments utilize a highly collimated non-coherentLED as a specular light source. As also seen in the embodiment of FIG.1, angles A and B of beams 225 and 224, respectively are achieved bypositioning of the light sources 222 and 222 such that the beams areemitted from those sources at the desired orientation. In otherembodiments, light source 223 and/or 222 may have a differentorientation within mouse 200. In such an embodiment, the desired angle Aand/or the desired angle B are achieved by routing the light sourceoutput through one or more light guides, prisms and/or other opticalelements.

FIG. 2 shows tracking surface 218. In this example, the tracking surface218 contains at least two types of surfaces (90,91). Surface 90 is aglossy surface having a high reflectivity. Surface 91 is a surface thatis less reflective or glossy than surface 90. Grazing illumination mayprovide improved imaging over surface 91 whereas specular illuminationor illumination striking the tracking surface 218 at a more normal anglemay provide improved imaging over surface 90. As an input device istracked over tracking surface 218 (portion 90 or 91), mouse 200 providesillumination using a method that is better suited for the type ofsurface being tracked. For example, if the mouse 200 is tracked overportion 91, the first light source 223 provides grazing illumination ata relatively shallow angle to the tracking surface. Conversely, when themouse 200 of FIG. 1 is tracked over portion 90, the second light source222 provides illumination via a light beam that strikes the portion 90at a more normal (i.e., more perpendicular) angle to the trackingsurface 218 as compared to the angle of the first light source 223.

A light source (and a corresponding tracking algorithm to be performedby IC 228) can be chosen in various manners. In some cases, a lightsource and corresponding tracking algorithm may be selected based on aselection criteria. The selection criteria may be any criteria or signalsuch as a manual input (e.g., a switch position or other user input) orautomatic selection based on other selection criteria pertaining to thedevice or tracking surface. In some cases, the illumination method (andcorresponding tracking algorithm) used in the mouse 200 may be changedor switched manually. In particular, the mouse 200 may include a manualswitch 255 which a user may activate to switch from one light source andtracking algorithm to another light source and tracking algorithm.Manual switching can be used, e.g., when a user intends to operate mouse200 on a surface that is entirely composed of a surface type suitablefor grazing illumination (e.g., an unvarnished wood desk top) or on asurface that is entirely composed of a surface type suitable forspecular illumination (e.g., a varnished and polished wood surface).Placing switch 255 in one position causes mouse 200 to solely illuminatewith the first light source 223 and to use a corresponding trackingalgorithm. Placing switch 255 in a second position causes mouse 200 toilluminate solely with second light source 222 and to use anothercorresponding tracking algorithm. Placing switch 255 in a third positioncauses mouse 200 to automatically select the best illumination methodand corresponding tracking algorithm. As described in more detail below,mouse 200 can automatically select the best illumination source (andcorresponding tracking algorithm) based on attributes of light receivedby array 226. In other embodiments, mouse 200 does not have a switch255, and illumination method is always selected automatically. In yetother embodiments, no automatic illumination selection is performed. Insome such embodiments, switch 255 is only a two position switch, witheach position selecting one of two available illumination methods.

FIG. 3 is a partially schematic block diagram of IC 228. Array 226 of IC228 includes a plurality of pixels p. Each pixel p is a photodiode orother photosensitive element which has an electrical property thatvaries in relation to the intensity of received light. For simplicity,only nine pixels are shown in FIG. 3. Array 226 may have many morepixels, and those pixels may be arranged in a variety of different ways.At multiple times, each pixel outputs a signal (e.g., a voltage). Theraw pixel output signals are amplified, converted to digital values andotherwise conditioned in processing circuitry 34. Processing circuitry34 then forwards data corresponding to the original pixel output signalsfor storage in RAM 36. Computational logic 38 then accesses the pixeldata stored in RAM 36 and calculates motion based on that data.Computational logic 38 employs one algorithm for tracking motion whenillumination is from first source 223 and a different motion trackingalgorithm illumination is from second source 222.

In one example, the same array may be used for detecting andcharacterizing different types of illumination. For example, if a firstlight source provides grazing illumination and a second light sourceprovides specular illumination, one array may be used to detect bothtypes of illumination. In another example, a different array may be usedfor detecting and characterizing different types of illumination. Forexample, a first light source may provide grazing illumination that isdetected by a first sensor and a second light source may providespecular illumination that is detected by a second sensor. In thisexample, the first sensor does not detect specular illumination and thesecond sensor does not detect the grazing illumination. In yet anotherexample, both any number of types of illumination may be detected by anarray in which the array determines a configuration corresponding to thereceived illumination or light. For example, an array may have acorresponding configuration including pixel size, number of pixels, etc.in which a particular type of illumination may be detected andrecognized. Various algorithms are known for tracking motion basedgrazing illumination images and for tracking motion based on specularillumination images, and the details of such algorithms are thereforeomitted. Similarly, and because numerous specific circuits for capturingvalues from a set of photosensitive pixels are known in the art,additional details of IC 228 are also omitted. Notably, FIG. 3 generallyshows basic elements of circuitry for processing, storing and performingcomputations upon signals obtained from an array. Numerous otherelements and variations on the arrangement shown in FIG. 3 are known topersons skilled in the art. For example, some or all of the operationsperformed in processing circuitry 34 could be performed within circuitelements contained within each pixel. At least some herein-describedembodiments are directed to details of calculations, performed withincomputational logic 38, for determining which illumination source (andcorresponding tracking algorithm) to select. Adaptation of knowncircuits to include the necessary pixel arrangements and perform suchcalculations (together with motion tracking algorithm computations andother computations) are within the routine abilities of persons ofordinary skill in the art once such persons possess the informationprovided herein.

As also shown in FIG. 3, light sources 222 and 223 are controlled by IC228. Specifically, IC 228 selectively activates and deactivates lightsources 222 and 223. For simplicity, FIGS. 1 and 3 only illustrate twolight sources; however, any number of light sources may be controlled byIC 228. The computational logic 38 selects the light source andactivates the selected light source to provide illumination to thetracking surface. In one example, light received and detected at array226 is processed in the processing circuitry 34 so as to determinewhether to continue using a currently active illumination source orwhether to change to another source. Based on attributes of the receivedlight, the computational logic 38 selects the appropriate one of lightsources 223 or 222. Selection and activation or deactivation of lightsources based on input signals received at the array 226 are describedin more detail below.

FIG. 4 is a partially schematic diagram of array 226 taken from theposition indicated in FIG. 1. For convenience, pixels in array 226 arelabeled p(r,c) in FIG. 4 where r and c are (respectively) the indices ofthe row and column where the pixel is located relative to the x and yaxes. In the embodiment of FIGS. 1 through 4, array 226 is a q by qarray, where q is an integer. The unnumbered squares in FIG. 4correspond to an arbitrary number of additional pixels. In other words,and notwithstanding the fact that FIG. 4 literally shows a ten pixel byten pixel array, q is not necessarily equal to ten in all embodiments.Indeed, array 26 need not be square. In other words, array 26 could be aq by q′ array, where q≠q′.

As set forth above, the mouse 200 of FIG. 1 is capable of switchingmanually or automatically from one light source to another light source.The mouse 200 may track a tracking surface 218 containing multipledifferent types of surfaces as illustrated in FIG. 2. The trackingsurface 218, for example, may contain a first portion 91 that is nothighly reflective and a second portion 90 that is highly reflective.When the mouse 200 tracks over the first portion 91 of the work surface218, the first light source 223 may produce a light incident on the worksurface 218 for grazing illumination. Hence, the first light source 223directs a beam 225 at the first portion 91 of work surface 218 visiblethrough opening 214. When the mouse is tracked over the second portion90 of the tracking surface 218, the mouse 200 may deactivate the firstlight source 223 and activate the second light source 222 such thatillumination is now provided by the second light source at a more normal(i.e., closer to perpendicular) angle with respect to the trackingsurface 218. In the embodiment of FIGS. 1-4, only one of light sources222 and 223 is emitting light at a time (except for brief overlapperiods when switching between light sources). Which light source emitslight is determined by attributes of light received in array 226.

One example of such an attribute is the amount of light (or theillumination level) received by array 226. The illumination level, whichcan be gauged based on the overall output of pixels in array 226, maydepend on many factors (e.g., intensity of the illumination source,exposure time, the optical path between the illumination source and thearray, etc.). However, for a given mouse having a known combination ofcomponents and activating a known light source for a known period oftime, illumination level can generally be used to indicate what type ofsurface is being struck with that light source. For example, a laserstriking a glossy or highly reflective surface will usually result in ahigher illumination level than might result from a laser striking anon-glossy/lesser reflective surface. Similarly, an LED striking a roughsurface at a grazing angle will usually result in a higher illuminationlevel than might result from that LED striking a glossy or highlyreflective surface.

The illumination source may be pre-determined and set to a fixed value,and the sensor may adjust. In this example, the sensor may adjust bychanging the integration time. The amount of light each illuminationsource and corresponding sensor receives may thus depend on theconfiguration. In one example, an LED-based glancing illumination methodhas shallow angle illumination, with the sensor positioned directlyabove the illuminated area (normal to surface). In this case, arelatively rough and diffuse surface may appear bright because the lightis being more uniformly scattered from the surface in all directions.If, however, the surface is relatively shiny, then much of the lightstriking the surface may be reflected directly along the specular angle.In this case, relatively little light reflects straight up to thesensor. Therefore, in this case, the surface may appear darker to thesensor (from less light received). In yet another example, theillumination source and sensor may be positioned directly on thespecular angle. In this case, relatively shiny surfaces may get the mostlight back to the sensor and diffuse surfaces may get the least lightback to the sensor.

A predetermined threshold may be set such that when an illuminationlevel is above the predetermined threshold with a selected light source,that light source may continue to be used by the mouse or other inputdevice to image a tracking surface. For example, if the illuminationlevel is above the predetermined threshold level when an LED lightsource is used for grazing illumination, the use of the LED light sourcemay continue to be used in tracking the surface. When the illuminationlevel drops below the predetermined threshold, however, the mouse orother input device may switch from the LED light source to a differentlight source so that the illumination threshold may increase to a levelabove the predetermined threshold. For example, if an LED light sourceis being used in grazing illumination and the illumination level dropsbelow the predetermined threshold level, then the mouse or other inputdevice may switch to a laser light source for specular imaging such thatthe illumination level rises above the predetermined threshold level.The actual threshold values may be different for different types ofillumination (e.g., different for LED illumination, laser illumination,etc.). Also the threshold values may be set to any desired level so thatimproved imaging may be accomplished on a variety of different types ofsurfaces.

Image contrast is another attribute of light received at array 226 whichcan be used to determine light source and tracking method. In oneexample, image contrast may be determined using auto-correlation. Forexample, FIG. 4 illustrates an example of an array of pixels arranged inX rows and Y columns. For illustration purposes, the pixels are labeledin FIG. 4 as p(x,y), where the x and y parameters indicate the row andcolumn of the pixel, respectively, within the array.

As illustrated in the example of FIG. 4, each non-edge pixel (i.e., eachpixel that is not on an edge of the array) is surrounded by 8neighboring pixels. For example, pixel p(2,2) in FIG. 4 is surrounded bypixels p(1,1), p(1,2), p(1,3), p(2,1), p(2,3), p(3,1), p(3,2), andp(3,3). In general, and as shown in FIG. 4 for an arbitrary location inarray 226, each non-edge pixel p(x,y) is surrounded by eight otherpixels as follows:

p(x − 1, y − 1) p(x − 1, y) p(x − 1, y + 1) p(x, y − 1) p(x, y) p(x,y + 1) p(x + 1, y − 1) p(x + 1, y) p(x + 1, y + 1)

An adjacent pixel or neighboring pixel may be described generally byp(x+i, y+j), where i and j can each be −1, 0, or 1. A difference betweenany pixel in an image frame and one of its neighboring pixels may bedetermined by Equation 1.Diff(i,j)=Out[p(x,y)]−Out[p(x+i, y+j)]  Equation 1

In Equation 1, Out[p(x,y)] and Out[p(x+i, y+j)] are the digitizedoutputs of pixels p(x,y) and p(x+i , y+j), respectively. In at least oneembodiment, a difference between pixel outputs alternately may beexpressed as an absolute value of a difference (shown in Equation 2) oras a difference squared (Equation 3).Diff(i,j)=ABS[Out[p(x,y)]−Out[p(x+i, y+j)]]  Equation 2Diff(i,j)=[Out[p(x,y)]−Out[p(x+i, y+j)]]²   Equation 3

Using Equation 2 or Equation 3, the differences between all non-edgepixels in array 226 of FIG. 4 and their respective i,j neighbors can beexpressed as shown in Equation 4 or Equation 5.

$\begin{matrix}{{A\;{C\left( {i,j} \right)}} = {\sum\limits_{x = 2}^{q - 1}{\sum\limits_{y = 2}^{q - 1}\left\lbrack {{ABS}\left( {{{Out}\mspace{14mu}\left( {p\left( {x,y} \right)} \right)} - {{Out}\mspace{14mu}\left( {p\left( {{x + i},{y + j}} \right)} \right)}} \right)} \right\rbrack}}} & {{Equation}\mspace{14mu} 4} \\{{A\;{C\left( {i,j} \right)}} = {\sum\limits_{x = 2}^{q - 1}{\sum\limits_{y = 2}^{q - 1}\left\lbrack {{{Out}\mspace{14mu}\left( {p\left( {x,y} \right)} \right)} - {{Out}\mspace{14mu}\left( {p\left( {{x + i},{y + j}} \right)} \right)}} \right\rbrack^{2}}}} & {{Equation}\mspace{14mu} 5}\end{matrix}$

AC in Equations 4 and 5 represents an autocorrelation value. Equations 4and 5 are only two examples of manners in which an autocorrelationvalues may be obtained. Other techniques for calculating anautocorrelation value could be employed.

Using all combinations of i and j between −1 and 1, eightautocorrelation values may be obtained for a given image frame:AC(−1,−1), AC(−1,0), AC(−1,1), AC(0,−1), AC(0,1), AC(1,−1), AC(1,0),AC(1,1). Because AC(0,0) is a comparison of a pixel to itself, it wouldbe expected to have a value of zero, and is thus ignored. The AC valuesare then used to calculate image contrast. For example, an image withpoor or low contrast would have relatively low AC values; an image withhigh contrast would have relatively high AC values. A contrast metriccan be calculated with the AC values. As one example, the contrastmetric may be an average of the eight AC values associated with a givenimage.

In at least some embodiments, the choice of illumination source isdetermined by evaluating both the illumination level and the imagecontrast. FIG. 5 is a flowchart that illustrates an example ofdetermining an illumination method based on the illumination level andcontrast of a received image. In STEP 701, a default illumination source(and corresponding tracking algorithm) is selected. For example, thedefault illumination source in mouse 200 may be set to LED 223, and atracking algorithm corresponding to grazing illumination employed.Alternatively, the default illumination source may be set to VCSEL 222,and a tracking algorithm corresponding to specular illuminationemployed.

In STEP 702, the illumination level is determined In STEP 703, thedetected illumination level is compared to a predetermined thresholdvalue for illumination. If the value from STEP 702 is higher than thepredetermined threshold value, then the default illumination source andthe corresponding tracking algorithm (STEP 704) may be used. Afterwaiting a delay time T (STEP 708), the illumination level may bedetected again (STEP 702) to determine if the illumination level haschanged relative to the predetermined threshold value. If the value islower than the predetermined threshold, then the contrast is determined(STEP 705). In determining the contrast, AC values may be determined andused in a metric as described above. The contrast is compared to athreshold value (STEP 706). If the contrast is less than a predeterminedthreshold value, then the illumination source and tracking algorithm arechanged (STEP 707). However, if the contrast is higher than thepredetermined contrast threshold value, then use of the defaultillumination method and tracking algorithm are continued (STEP 704).After the illumination source and tracking algorithm are changed in STEP707, further changes in the illumination level may be detected (STEP702). As illustrated in this example, after waiting a delay time T (STEP709), further changes in the illumination level may be detected.

In an alternative method, the illumination method and tracking algorithmmay be switched when either the illumination level or the contrast fallsbelow a predetermined threshold. FIG. 6 is a flowchart illustrating oneexample of such a method. In STEP 801, a default illumination method andcorresponding tracking algorithm are selected. Based on the defaultillumination method, the illumination level of a tracking surface isdetected (STEP 802). For example, if an LED and grazing illumination isselected as the default illumination method, the illumination of thetracking surface based on grazing illumination from the LED light sourceis calculated. The calculated illumination level is compared to apredetermined threshold illumination level (STEP 803). If the calculatedillumination level is less than the predetermined threshold illuminationlevel, then the illumination method and tracking algorithm are switchedto an alternative illumination method and tracking algorithm (STEP 804).After waiting a delay time T (STEP 808), subsequent changes in theillumination method may be detected (STEP 802).

If in STEP 803 the detected illumination level based on the defaultillumination method is higher than the predetermined threshold, then theimage contrast of the tracking surface is calculated (STEP 805) andcompared to a predetermined threshold contrast value (STEP 806).Contrast can be calculated using autocorrelation as described above. Ifthe calculated image contrast is less than the predetermined thresholdcontrast levels, then the illumination method and tracking algorithm arechanged (STEP 804). After waiting a delay time T (STEP 808), the processmay repeat to determine any subsequent changes in illumination level(STEP 802) and/or contrast level (STEP 805). Alternatively, if thecalculated contrast level is higher than the predetermined contrastthreshold, then use of the default illumination method and itscorresponding tracking algorithm are continued (STEP 807). After waitinga delay time T (STEP 808), the process may repeat to detect otherchanges in illumination level and/or contrast level.

In at least some embodiments, a mouse such as mouse 200 implements thealgorithm of FIG. 5. In at least some other embodiments, a mouse such asmouse 200 implements the algorithm of FIG. 6. In some embodiments, thealgorithm of FIG. 5 or FIG. 6 may be implemented in the IC 228 of FIGS.1 or 3.

FIG. 7 is a block diagram illustrating an example of an apparatus forselecting an illumination method and corresponding tracking algorithm inan input device such as a mouse. The apparatus may include anillumination sensor 901 that receives an image input from a trackingsurface. In some cases, the illumination sensor 901 may be contained inan array similar to array 226 illustrated in FIG. 3. The array mayfurther be contained in an IC similar to IC 228 as illustrated in FIG. 3that may further contain computational logic and processing circuitrysuch as computational logic 38 and processing circuitry 34, illustratedin FIG. 3. The computational logic and processing circuitry maydetermine and select a light source and corresponding trackingalgorithm. Light may be reflected from the mouse to the tracking surfaceand may be reflected back to the mouse and received at the illuminationsensor 901. In addition, the apparatus may optionally include an input904 for receiving a selection criteria that may be used in selecting alight source and corresponding tracking algorithm. For example, theinput 904 may include a switch that may be similar to the switch 255illustrated in FIG. 3. Alternatively, the input 904 may include a userinterface in which a user may input a desired threshold value for theillumination level. For example, a user may input a desired thresholdvalue via input 904. The desired threshold value may be stored instorage 905. In another example, a predetermined threshold value may bepre-stored in storage 905 without user input.

The input image illumination received at the illumination sensor 901 maybe compared to the illumination threshold value in storage 905 by acomparator module 902. In some examples, the comparator module may behoused in an IC, such as an IC similar to IC 228 illustrated in FIG. 3.For example, the comparator module 902 may be contained in computationallogic on the IC similar to the computational logic 38 illustrated inFIG. 3. The comparator module 902 may receive the image illuminationfrom the illumination sensor 901 and retrieves the stored illuminationthreshold from storage 905. The comparator module 902 further determinesif the received image illumination is less than or greater than thestored illumination threshold value. The comparator module 902 may alsoprovide the comparison data to the illumination controller 903. Theillumination controller 903 may also be contained in the computationallogic 38 the IC. Based on the comparison data from the comparator module902, the illumination controller 903 may control any number or type oflight sources. In this example, the illumination controller 903 controlsa plurality of light sources (e.g., Light Source A 906, Light Source B907 and Light Source N 908). In some cases, the different light sourcesmay be of different types and/or may be positioned at different angleswith respect to the tracking surface.

In one example, light is provided to the tracking surface via lightsource A 906. Also, the comparator module 902 determines that theillumination threshold value from storage 905 is less than theillumination level of the received image at the illumination sensor 901which corresponds to the reflected light from light source A 906. Basedon this determination at the comparator module 902, the illuminationcontroller 903 may control light source A 906 to provide light to thetracking surface and may also select a tracking algorithm correspondingto light source A 906. Also, the illumination controller 903 may controlthe other light sources (i.e., light source B 907-light source N 908)not to provide light to the tracking surface (i.e., the illuminationcontroller 903 may ensure that light source B 907 and light source 908remains in an off state).

In another example, light is provided to the tracking surface via lightsource A 906, however, the comparator module determines that theillumination threshold value from storage 905 is greater than theillumination level of the received image at the illumination sensor 901which corresponds to the reflected light from light source A 906. Basedon this determination at the comparator module 902, the illuminationcontroller 903 may control light source A 906 to terminate (i.e., mayturn off light source A 906) and may control a different light source,such as light source B 907 or light source N 908 to emit light to thetracking surface (i.e., turn on a different light source). Also, theillumination controller 903 may select a tracking algorithmcorresponding to the controlled light source (e.g., light source B 907or light source N 908). For example, if light source A 906 emits lightto the tracking surface which is reflected back to the mouse andreceived at illumination sensor and the received illumination signalsreceived at the illumination sensor 901 are determined to be less thanthe threshold illumination values in storage 905 by the comparatormodule 902, then the illumination controller 908 (based on input fromthe comparator module 902) may select a different light source such aslight source B 907 or light source N 908, and corresponding trackingalgorithm. The selection of an alternate light source and correspondingtracking algorithm may be based on attributes of the light source. Forexample, the attributes may include illumination level, the type oflight source or the position of the light source in the mouse relativeto the tracking surface. Also, in addition to the illumination level,the light source may be selected based on contrast level of theillumination. For example, the light source may be selected based onboth the illumination level and the contrast level of the light source.

FIG. 8 is a block diagram illustrating another example of an apparatusfor selecting an illumination method and corresponding trackingalgorithm in an input device such as a mouse. The apparatus may includean image sensor 1001 that receives an image input from a trackingsurface. The image sensor 1001 may include an array, for example, anarray similar to array 226 illustrated in FIG. 3. Also, the array may belocated on an IC. In one example, the IC may be similar to IC 228illustrated in FIG. 3. The image received may include various attributesincluding contrast differences between pixels in the image. Light may beemitted from the mouse to the tracking surface and may be reflected backto the mouse and received at the image sensor 1001 at an array. Inaddition, the apparatus may optionally include an input 1011 forreceiving a selection criteria for selecting a light source andcorresponding tracking algorithm. For example, the input 1011 mayinclude a switch (e.g., a switch similar to switch 255 illustrated inFIG. 3) for manually inputting a desired light source. Alternatively,the input 1011 may include a user interface for receiving a desiredthreshold value for an image contrast or a threshold value ofdifferences in pixel contrast between different pixels such as pixelsthat are adjacent to each other. For example, a user may input a desiredthreshold value via input 1011. The desired threshold value may bestored in storage 1012. Alternatively, the threshold value may be storedin storage 1012 without input from the user (e.g., factory-installed).

The image sensor 1001 may receive the input image (e.g., from a worksurface) and may provide the input image to the pixel selector 1002. Thepixel selector 1002 may select a pixel from the array of pixels in theimage. Information on the contrast of the selected pixel and/or pixelsthat adjoin, abut, are neighboring, or are adjacent to the selectedpixel may also be obtained. Image contrast may be further characterizedin the pixel contrast module 1004 which may determine the contrastdifferences between selected pixels and any of the selected pixels'neighboring pixels.

The comparator module 1003 may compare the pixel contrast to that ofpixels that are next to, adjacent, or neighboring the selected pixel. Insome examples, the comparator module 1003 may be located on an IC suchas an IC similar to IC 228 illustrated in FIG. 3. In some examples, theIC may further include computational logic such as computational logic38 illustrated in FIG. 3. Alternatively, the comparator module may belocated on the array itself and may be further controlled by processingcircuitry similar to processing circuitry 34 illustrated in FIG. 3. TheIC may further include an auto correlator 1005. For example, the autocorrelator 1005 may be part of the computational logic or processingcircuitry of the IC. The auto correlator 1005 may receive pixel contrastdata from the comparator module 1003 and may determine, based on thereceived data, at least one autocorrelation (AC) parameter. The ACcomparator 1006 may compare the AC parameters or contrast levels of thereceived image with a predetermined contrast threshold value. Thecontrast threshold value may be stored in storage 1012 in oneembodiment. For example, a user may optionally input a desired thresholdvalue via input 1011 and the input threshold value may be stored instorage 1012. Based on the image contrast, the illumination controller1007 may control at least one light source. In the example illustratedin FIG. 8, the illumination controller 1007 selects a correspondinglight source from a plurality of light sources (i.e., Light Source A1008, Light Source B 1009, . . . , Light Source N 1010, in the exampleillustrated in FIG. 8). Also, as set forth above, the light source maybe selected based on both the contrast level and the illumination levelof the received light.

FIG. 9 illustrates another embodiment of a mouse containing at least twoseparate light sources in which each light source emits an incidentlight at a different portion of the work surface. In this example, anLED light source 601 emits grazing illumination 603 onto the trackingsurface 602 to provide a reflected light beam 604. The reflected lightbeam 604 is received at an array 605 of an IC 610. Also, the inputdevice may further contain a separate light source 606 for emitting alight beam 607 at the tracking surface 602. The light beam 607 strikesthe tracking surface 602 at a different point on the tracking surface ascompared to the light beam 603 from the LED light source 601. Thereflected beam of light 609 from the incident light beam 607 from theseparate light source 606 may be received at array 608 of IC 611. Thearray 608 is positioned at a different location in the device relativeto the array 605. Also, the separate light source 606 may be oriented tothe tracking surface 602 at a different angle as compared to the LEDlight source 601. For example, the LED light source 601 may bepositioned to emit a light beam 603 that is approximately 20 degrees orless from the work surface 602 and the separate light source 606 may bepositioned to emit a light beam 607 that is approximately 20 degrees orless from a vertical or perpendicular orientation.

IC 610 and IC 611 may each contain logic for determining which array andlight source (601 or 606) to use. The determination of which lightsource to use and which corresponding tracking algorithm to use may bebased on any selection criteria including, for example, a manual input(e.g., a switch position) or selection criteria based on the status ofthe device or tracking surface 602. Selection of a light source andcorresponding tracking algorithm may also be determined automatically,for example, based on attributes of the tracking surface 602. Forexample, if the tracking surface 602 is highly reflective or glossy,light source 606 and corresponding tracking algorithm may be selected.If the tracking surface 602 is not highly reflective or non-glossy, thenlight source 601 and corresponding tracking algorithm may be selected.As set forth above, logic contained in the IC 610 or the IC 611 mayperform the selection based on tracking surface attributes such asillumination level or contrast level. In addition, one IC may controlmore than one array. Thus, in one example, more than one array isincluded on the IC for detecting light and each of the more than onearrays may be controlled by a single IC. In another example, more thanone array is included on the IC for detecting light and each of the morethan one arrays may be controlled by any number of ICs (e.g., 2, 3, 4,5, 6, etc.). In yet another example, one IC may control all of thearrays included on the IC.

It is understood that aspects of the present invention can take manyforms and aspects. The embodiments shown herein are intended toillustrate rather than to limit the invention, it being appreciated thatvariations may be made without departing from the spirit of the scope ofthe invention. Although illustrative aspects of the invention have beenshown and described, a wide range of modification, change andsubstitution is intended in the foregoing disclosure and in someinstances some features of the present invention may be employed withouta corresponding use of the other features. Accordingly, it isappropriate that the appended claims be construed broadly and in amanner consistent with the scope of the invention. As used herein(including the claims), “light” includes emissions in the visible and/ornon-visible portions of the electromagnetic spectrum. In the claims,various portions are prefaced with letter or number references forconvenience. However, use of such references does not imply a temporalrelationship not otherwise required by the language of the claims.

1. A method for modifying illumination sources for an input device, themethod comprising: activating a first illumination source to illuminatea surface external to the input device with light emitted from the firstillumination source; detecting a first illumination level from the firstillumination source; comparing the first illumination level to a firstthreshold value; when the first illumination level exceeds the firstthreshold value, using the first illumination source for the inputdevice; and otherwise, when the first illumination level is less thanthe first threshold value: detecting a contrast level from the firstillumination source; comparing the contrast level to a second thresholdvalue calculated as an averaged value of a plurality of autocorrelationvalues for only non-edge pixels obtained from a captured image of thesurface external to the input device; when the contrast level is lessthan the second threshold value, using a second illumination source forthe input device; and otherwise, when the contrast level is greater thanthe second threshold value, using the first illumination source for theinput device.
 2. The method of claim 1, further comprising comparing thecontrast level with the plurality of autocorrelation values calculatedas one of an averaged absolute value of a difference between adjacentnon-edge pixel outputs and an averaged value of a difference squaredbetween adjacent non-edge pixel outputs.
 3. The method of claim 2,wherein${{A\;{C\left( {i,j} \right)}} = {\sum\limits_{x = 2}^{q - 1}{\sum\limits_{y = 2}^{r - 1}\left\lbrack {{ABS}\left( {{{Out}\left( {p\left( {x,y} \right)} \right)} - {{Out}\left( {p\left( {{x + i},{y + j}} \right)} \right)}} \right)} \right\rbrack}}},$AC(i,j) is an autocorrelation value for non-edge pixels in an arrayrelative to the i,j neighbors of said non-edge pixels, i=−1, 0 or 1,j=−1, 0 or 1, the array has q pixels in an X direction and r pixels in aY direction, with q and r both being greater than three, p(x,y) is apixel located at location x, y in the array, p(x+i, y+j) is a neighborpixel of pixel p(x,y), Out(p(x,y)) is an output value corresponding topixel p(x,y), Out(p(x+i, y+j)) is an output value corresponding to pixelp(x+i, y+j), and wherein the averaged absolute value is calculated basedon values for AC(−1, −1), AC(−1,0), AC(−1,1), AC(0,−1), AC(0,1),AC(1,−1), AC(1,0) and AC(1,1).
 4. The method of claim 2, wherein${A\;{C\left( {i,j} \right)}} = {\sum\limits_{x = 2}^{q - 1}{\sum\limits_{y = 2}^{r - 1}\left\lbrack {{{Out}\left( {p\left( {x,y} \right)} \right)} - {{Out}\left( {p\left( {{x + i},{y + j}} \right)} \right)}} \right\rbrack^{2}}}$AC(i,j) is an autocorrelation value for non-edge pixels in an arrayrelative to the i,j neighbors of said non-edge pixels, i=−1, 0 or 1,j=−1, 0 or 1, the array has q pixels in an X direction and r pixels in aY direction, with q and r both being greater than three, p(x,y) is apixel located at location x, y in the array, p(x+i, y+j) is a neighborpixel of pixel p(x,y), Out(p(x,y)) is an output value corresponding topixel p(x,y), Out(p(x+i, y+j)) is an output value corresponding to pixelp(x+i, y+j), and wherein the averaged value is calculated based onvalues for AC(−1,−1), AC(−1,0), AC(−1,1), AC(0,−1), AC(0,1), AC(1,−1),AC(1,0) and AC(1,1).
 5. The method of claim 1, wherein one of the firstand second illumination sources is a grazing source illuminating thesurface at a first angle, the other of the first and second illuminationsources is a specular source illuminating the surface at a second angle,and the second angle is closer to a normal from the surface than thefirst angle.
 6. The method of claim 5, wherein at least one of the firstand second illumination sources is a laser.
 7. The method of claim 6,wherein one of the first and second sources is the laser and the otherof the first and second sources is a light emitting diode.
 8. The methodof claim 1, wherein one of the first and second sources is a laser andthe other of the first and second sources is a light emitting diode. 9.The method of claim 1, wherein a light beam from the first illuminationsource and a light beam from the second illumination source arenon-coincident.
 10. A method for modifying illumination sources for aninput device, the method comprising: activating a first illuminationsource to illuminate a surface external to the input device with lightemitted from the first illumination source; detecting a firstillumination level from the first illumination source; comparing thefirst illumination level to a first threshold value; when the firstillumination level exceeds the first threshold value, using the firstillumination source for the input device, and otherwise, using a secondillumination source for the input device; detecting a contrast levelfrom the first illumination source; comparing the contrast level to asecond threshold value calculated as an averaged value of a plurality ofautocorrelation values for only non-edge pixels obtained from a capturedimage of the surface external to the input device; and when the contrastlevel exceeds the second threshold value, using the first illuminationsource for the input device, and otherwise, using the secondillumination source for the input device.
 11. The method of claim 10,further comprising comparing the contrast level with the plurality ofautocorrelation values calculated as one of an averaged absolute valueof a difference between adjacent non-edge pixel outputs and an averagedvalue of a difference squared between adjacent non-edge pixel outputs.12. The method of claim 11, wherein${{A\;{C\left( {i,j} \right)}} = {\sum\limits_{x = 2}^{q - 1}{\sum\limits_{y = 2}^{r - 1}\left\lbrack {{ABS}\left( {{{Out}\left( {p\left( {x,y} \right)} \right)} - {{Out}\left( {p\left( {{x + i},{y + j}} \right)} \right)}} \right)} \right\rbrack}}},$AC(i,j) is an autocorrelation value for non-edge pixels in an arrayrelative to the i, j neighbors of said non-edge pixels, i=−1, 0 or 1,j=−1, 0 or 1, the array has q pixels in an X direction and r pixels in aY direction, with q and r both being greater than three, p(x,y) is apixel located at location x, y in the array, p(x+i,y+j) is a neighborpixel of pixel p(x,y), Out(p(x,y)) is an output value corresponding topixel p(x,y), Out(p(x+i,y+j)) is an output value corresponding to pixelp(x+i,y+j), and wherein the averaged absolute value is calculated basedon values for AC(−1,−1), AC(−1,0), AC(−1,1), AC(0,−1), AC(0,1),AC(1,−1), AC(1,0) and AC(1,1).
 13. The method of claim 11, wherein${A\;{C\left( {i,j} \right)}} = {\sum\limits_{x = 2}^{q - 1}{\sum\limits_{y = 2}^{r - 1}\left\lbrack {{{Out}\left( {p\left( {x,y} \right)} \right)} - {{Out}\left( {p\left( {{x + i},{y + j}} \right)} \right)}} \right\rbrack^{2}}}$AC(i,j) is an autocorrelation value for non-edge pixels in an arrayrelative to the i, j neighbors of said non-edge pixels, i=−1, 0 or 1,j=−1, 0 or 1, the array has q pixels in an X direction and r pixels in aY direction, with q and r both being greater than three, p(x,y) is apixel located at location x, y in the array, p(x+i,y+j) is a neighborpixel of pixel p(x,y), Out(p(x,y)) is an output value corresponding topixel p(x,y), Out(p(x+i,y+j)) is an output value corresponding to pixelp(x+i,y+j), and wherein the averaged value is calculated based on valuesfor AC(−1,−1), AC(−1,0), AC(−1,1), AC(0,−1), AC(0,1), AC(1,−1), AC(1,0)and AC(1,1).
 14. The method of claim 10, wherein one of the first andsecond illumination sources is a grazing source illuminating the surfaceat a first angle, the other of the first and second illumination sourcesis a specular source illuminating the surface at a second angle, and thesecond angle is closer to a normal from the surface than the firstangle.
 15. The method of claim 14, wherein at least one of the first andsecond illumination sources is a laser.
 16. The method of claim 15,wherein one of the first and second sources is the laser and the otherof the first and second sources is a light emitting diode.
 17. Themethod of claim 10, wherein one of the first and second sources is alaser and the other of the first and second sources is a light emittingdiode.
 18. The method of claim 10, wherein a light beam from the firstillumination source and a light beam from the second illumination sourceare non-coincident.